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DESCRIPTION 

Attorney Docket No. 38195.69 

A VIDEO INPUT DEVICE FOR VIDEOPHONE SIGN LANGUAGE CONVERSATION 
WITH SIGN LANGUAGE, A VIDEO INPUT/OUTPUT A SSISTANCE DEVICE ¥GR 
CONVERSATION WITH SIGN LANGUAGE, A ND A SIGN LANGUAGE 
INTERPRETATION SYSTEM USING THE SAME 



TcchnicQl B ACKGROUND OF THE INVENTION 
1 . Field of the Invention 

The present invention relates to a video input device, 
a video input/output device and a sign language interpretation 
system using inc luding the same which are used by e^deaf-mute 
pcrgon persons to have a sign language conversation v^ith sign 
language — by — way — o€ — using communications means such as a 
videophone, and in particular , the present invention relates 
to a video input device for sign language conversation with 
sign language , a video input/output device for sign language 

conversation v^ith sign language and a sign language 

interpretation system using including the same which are 
preferable for example in transmitting u sed to transmit a video 
other than sign language while performing explanation by sign 
language . 
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Background 2. Description of the Related Art 

While sign language is important moans #e^^ 

communicationo to for communicating with a deaf-mute person, 
the picture quality of prior art videophones was poor and not 
sufficient for a conversation with sign language conversations 
between deaf-mute persons in remote locations. Nowadays , vjith 

^he advancGmont ift t4ie — With recent advancements in 

communications technology, the picture quality of a videophone 
has been greatly improved , thus allowing a conversation with . 
Accordingly, sign language conversation between deaf-mute 
persons in remote locations to be available at a is now practical 
iQvcl. and available. 

Fig. 1 3 shows a conceptual diagram of a conversation v/ith 
sign language conversation between deaf-mute persons by way 
of using a prior art videophone. In Fig. 13, a numeral 10 
represents a videophone terminal used by a deaf-mute person 
A and numeral 2 0 represents a videophone terminal used by a 
deaf-mute person B. The deaf-mute person A sets the videophone 
terminal 10 ee -such that his/her sign language will be picked 
upcaptured by an imaging section 10b and the sign language of 
the deaf-mute person B displayed in a video display section 
10a will be viewed. Similarly, the deaf-mute person B sets 
the videophone terminal 2 0 -sesuch that his/her sign language 
will be piclced up captured by an imaging section 20b and the 
sign language of the deaf-mute person A displayed in a video 



2 



display section 20a will be viewed. By doing so, the deaf-mute 
person A and the deaf-mute person B have a Gonvcroation with 
sign language convcroation via a videophone. While a cellular 
phone typo is used as a videophone terminal in this example, 
a desktop- type videophone terminal may be also used. 

Next, a GQQQ situation will be described where in which 
a deaf-mute person converses with a non-deaf-mute person by 
using a videophone terminal via a sign language interpreter. 
Such sign language interpretation is implemented by using^ for 
example_^ a multipoint connection unit which interconnects 
three or more videophone terminals to provide a^teleconf erence 
services. 

Fig. 14 is a conceptual diagram of a sign language 
interpretation service using a prior art multipoint connection 
unit. In Fig. 14, a^numeral 10 represents a videophone terminal 
for deaf-mute persons used by a deaf-mute person A (hereinafter 
referred to as a deaf-mute person terminal) , numeral 20 
represents a videophone terminal for non-deaf -mute persons 
used by a non-deaf -mute person B (hereinafter referred to as 
a non-deaf -mute person terminal), and numeral 3 0 represents 
a videophone terminal for sign language interpreters used by 
a sign language interpreter C (hereinafter referred to as a 
sign language interpreter terminal) . A — numcral Numeral 1 
represents a multipoint connection unit. 

The multipoint connection unit 1 accepts connections 
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from the terminals 1 0, 20, 30, receives a^video and audio 
transmitted from the terminals, synthesizes the received video 
and audio, and delivers the resulting video and audio to each 
terminal. Thus, a video obtained by synthesizing the videos 
from the terminal is displayed on the display screens (10a, 
20b, 30b) of the terminals. An audio obtained by synthesizing 
audios collected by the microphones of the headsets (20c, 30c) 
and the lilco is output to loudspeakers such as the headsets 
(20c, 30c) of the terminals. Synthesis of videos uses^ for 
example^ a four-way synthesis which equally synthesizes the 
videos of all parties engaged. The deaf-mute person A does 
not use audio input/output e esuch that the headset of the 
deaf-mute person terminal 1 0 is omitted and voice 
communications are provided only between the non-deaf -mute 
person and the sign language interpreter. ift — case — fefee 
onvironmcnt W here the environmental or background sound is 
collected by the deaf-mute person terminal 10 and transmitted^ 
or in cQSG where a helper is present with the deaf-mute person , 
a microphone or a headset may be provided. 

With this configuration, when the deaf-mute person A 
performs sign language, the sign language interpreter C watches 
the sign language of the deaf-mute person A and translates it 
into a voice. The non-deaf -mute person B listens to the voice 
of the sign language interpreter C to understand the sign 
language of the deaf-mute person A. When the non-deaf -mute 
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person B speaks, the sign language interpreter C listens to 
the voice of the non-deaf -mute person B and translates it into 
sign language. The deaf-mute person A watches the sign 
language of the sign language interpreter C to understand the 
speech of the non- deaf -mute person B. 

However, in a conversation between deaf-mute persons 
using a videophone or a conversation between a deaf-mute person 
and a non-deaf -mute person via sign language interpretation, 
the videophone terminal for deaf-mute persons must pick 
t^cagture the sign language of the deaf-mute person and 
transmit the video to the distant party while the deaf-mute 
person is performing sign language, -sesuch that the videophone 
terminal for deaf-mute persons cannot transmit other videos 
to the distant party. Thus, the deaf-mute person cannot 
transmit a video other than sign language while explaining the 
and explain the video by ^using sign language in a videophone 
conversation. 

In this wgy manner , while it is possible to transmit a 
target video while cxplaining and explain the video by v/ay of 
a- using voice in a videophone conversation between unimpaired 
persons, there is no way of transmitting a target video while 
explaining the oamo it is not possible to transmit a target 
video and explain the target video in a videophone conversation 
involving a deaf-mute person . As a result , explanation of the a 
target is imprecise e^? — and a spccdy short conversation is 
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difficult . 

A main object 

SUMMARY OF THE INVENTION 

To overcome the problems described above, preferred 

embodiments of the present invention is to provide a video input 
dcviGO for videophone sign language conversation with sign 

language, a video input /output assistance device #ei^ 

convcroation with sign language including the aamc , and a sign 
language interpretation system using the same 
enabling v ideophone sign language conversation assistance 
device to enable a deaf-mute person to transmit a target video 
other than sign language while performing explanation by sign 
1 anguage . 

Diocloaure According to a preferred embodiment of the 
Invention 

¥-hepre_sent_ invention dcGGribcd in — claim — 1 — ±-s — a video 

input , a videophone sign language conversation assistance 
device for used by a deaf-mute person to have a conversation 
with sign language comprising : — oign languagc with the use of 
a videophone includes hand imaging means for picking up sign 
language; — video acquioi t ion including waist fixing means #ei^ 
acquiring a video other than sign language; — video synthesis 
means for synthcsiging a sign language video piclced up by the 
sign language to be fixed at the waist of a deaf-mute person 
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for capturing images of the hands of the deaf-mute person to 
acquire a sign language video, sight line direction imaging 
means fixed to the head of the deaf-mute person and q video 
other than sign languagc a rranged to capture images of the area 
in the direction of the sight line of the deaf-mute person, 
video signal synthesis means for synthesizing a video signal 
acquired by the video acquisition hand imaging means-r and video 
transmisGion a video signal acquired by the sight line direction 
imaging means for transmitting the video , and a videophone 
connection means including a function to transmit a video 
signal synthesized by the video signal synthesis means-r 
charactGrizcd in that a to a videophone terminal, wherein the 
deaf-mute person can edd rinclude an explanation by sign language 
while transmitting a video other than sign language. — in the 
sight line direction. 

With this configuration, the deaf-mute person can 
precisely explain the target in the sight line direction, and 
thus^ a conversation with sign language can be Gpccded is sped 
up. 

The videophone connection means can be connected 

to a videophone of the cellular phone type. Thus, a deaf-mute 
person can transmit to the opponent party a video other than 
sign language with explanation by sign language added even 
while moving, which adds to the convenience to the deaf-mute 
person. 
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The invention dcgcribcd in claim 2 io the video input device 
#e^e — converoation with sign language — according to — claim — ^ 
charactcriged in that the oign language imaging means compriGeG 
waist fixing means to be fixed at the waist of the deaf mute 
person. — 

With this configuration, the sign language of the 

deaf-mute person is picked up under certain conditions and 
transmitted to the other end under certain conditions opponent 
party even when the deaf-mute person changes his/her position 

or orientationT thereby — enabling — stable — sign — language 

conversation. — This allows a stable conversation with sign 
language - 

The — invention — described in claim3 is the video — input 

device for conversation v^ith sign language according to claim 
A — e^? — 3-7 — charactcriged in that — the video transmission means 
comprises — videophone — connection means — to be connected to — a 
videophone terminal . 

With this configuration, — the deaf mute person can use 

a general purpose videophone terminal to transmit a video other 
than sign language with explanation by sign language added. 

In particular, — t - he videophone conneet - ioB means — can bo 

connected to a videophone of the cellular phone type so that 
— deaf mute person — can transmit — a video — other — than — sign 
language with explanation by sign language added while on the 
road. — This adds to the convenience to the deaf mute person. 
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The invention dcgcribcd in claim 4 io the video input 

dcvico for convoroation with oign language according to claim 
■3-7 — characterized — in — that — y^^e — videophone — connection — means 
comprioco The video signal synthesis means preferably 
includes a function to synthesize a video signal captured by 
the sight line direction imaging means as a main window and 
a video signal acquired by the hand imaging means as a sub window 
in a Picture- in~Picture arrangement and a function to change 
the setting of the pes ition of the sub window 

The videophone sign language conversation assistance 

device preferably includes display means fixed to the head of 
the deaf-mute person for displaying a video received by the 
videophone terminal in front of the eyes of the deaf-mute person 
and simultaneously allowing the deaf-mute person to view the 
outer world including a target for sign language conversation, 
and the videophone connection means preferably includes a 
function to receive a video signal from the videophone terminal 
and supply the video signal to the display means. 

With this configuration, the deaf-mute person is able 

to include an explanation by sign language while transmitting 
a video other than sign language, as well as receive an 
explanation by sign language while viewing the outer world by 
freely shifting his/her sight line. The display means fixed 
in front of the deaf-mute person is preferably as small as 
possible so as not to obstruct viewing of the outer world. 
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The sight line direction imaging means and the display 



means are preferably molded into a frame which can 


be fixed 


to the ears and nose of said deaf-mute person. 




This enables the deaf-mute person to readily 


set the 


sight line direction imaging means and the display 


means at 


the optimum position in front of his/her eyes. 




The videophone connection means preferably 


includes 



radio communications means for performing radio 
communications with the videophone terminal. 



With — this — configuration, — it — ±s — ft© — longer — ncccsoary 

This eliminates the need to connect the video — input 

device for videophone sign language conversation v/ith sign 
language with assistance device to a videophone terminal via 
a cable, which greatly facilit ates oyotcm handling. 

The invention — dQOcribcd ifi — claim — B — a — video 

input/output device for According to another preferred 

embodiment of the present invention, a videophone sign language 
interpretation system connecting the videophone sign language 
conversation with — sign language — comprioing the video — input 
dovicQ for converDation with sign language according to claim 
3 or 4 , the videophone connection mcana including sign language 
video receiving means for receiving a sign language video being 
rocoivod by the videophone terminal, characterized assistance 
device according to the preferred embodiment described above 
with the video phone terminal of a deaf-mute person and 
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interconnecting the videophone terminal of the deaf-mute 
person, the videophone terminal of a non-deaf -mute person and 
the videophone terminal of a sign language interpreter in order 
to provide sign language interpretation by a sign language 
interpreter in that — t-he — video — input /output — dcvicQ — #ei^ 
converoQtion v^ith sign language — includoo display mcano — 
displaying a oign language video received by the sign language 
video receiving moans and fixing moans for fixing the display 

means in front of the — eyes of the deaf mute person, 

With this configuration, — the deaf mute person is able 

to add explanation by sign language v i zhilo transmitting a video 
other than sign language, — as well as get explanation by sign 
language — while viewing the — outer — world by — freely — shifting 
his/her sight line. — The display means fixed in front of the 
eyes of deaf mute person is preferably as small as possible 
so as not to hamper viewing of the outer vjorld. — 

?^h^ — invention — dcscribod — ±n — .claim — 6 — i-s — t-he — video 

input/output — device — #e^= — conversation — with — sign — language 
according to claim 5, charactcriged in that the display means 
comprises a convex lens which can substantially focus on a sign 
language — video — displayed — &ft — 14^« — display — means — when — t-he 
deaf mute person views — the outer world. — 

With this configuration, the deaf mute person can v^yatch 

a sign language video displayed on the display means v^yithout 
making focus adjustment of eyeballs when hc/shQ views the outer 
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world. — The sign language video dioplayod on the diaplay mcanG 
io enlarged by the convex long oo that the size of the display 
device is rcduQcd. 

?^he — invention — described — iR — claim — 7 — is — fe-he — video 

input /output — device — — conversation — with — sign — language 
according to claim 5 or 6; — charactcriged in that the fixing 
means has a frame structure which can bo fixed to the cars and 
nose of a deaf mute person, — 

With — this — c on f i gur a - t - i - on - fefee — deaf mute — person — ea«: 

readily set the display means at the optimum position in front 
of his/her - eyesy -' Which - adds to the convenience to the deaf mute 
person. — 

¥he — invention — described — ±¥t — claim — 8 — i-s — the — video 

input/output — device — #e^^ — conversation — with — sign — language 
according to any one of claims 5 through 1, characterised in 
that the video acquisition means includes target imaging means 
for pic]cing up a target other than sign language and the target 
imaging means — &s — fixed to the — fixing means and picks up the 
area in the viev^ing direction of the deaf mute person. — 

With — this — configuration, fehe — deaf mute — person — &aft 

transmit the target video with explanation by sign language 
added to the opponent party v/hilo directly viewing the target 
for explanation, — This allov^^s the deaf mute person to have a 
more speedy conversation with sign language in a more precise 
fashion. — 
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¥fe€ — invention deocribod in claim 9 — io a a videophone 

conversation between a deaf-mute person and a non-deaf -mute 
person, wherein the videophone sign language interpretation 
system intcrconncGt ing — a — vidcophonc includes terminal #e*^ 
deaf mute pcroonD used by the deaf mute person capable of using 
sign language to which is connected the video — input /output 
device for conversation M±th sign language according to any 
Ofte — e€ — claims — B — through — 8-7 — a — videophone — terminal — 
non deaf mute persons used by a non deaf mute person incapable 
of using — sign — language, — and a videophone — terminal — &&b — sign 
language interpreters used by a sign language interpreter in 
order to provide sign language interpretation in a conversation 
between a deaf mute person and a non deaf mute person over a 

videophone, characterized — i« — that — thre — system — comprises 

connection means equipped with a including a sign language 
interpreter registration table where the terminal number of 
the videophone terminal for sign language interpreters used 
by of a sign language interpreter is registered, the terminal 
connection means including a function to accept a call from 
the videophone terminal for of the deaf-mute person-s- or 
videophone terminal for of the non-deaf -mute persons-, a 
function to prompt a calling videophone terminal for which the 
call is accepted to enter the terminal number of the called 
terminal, a function to extract the terminal number of the 
videophone terminal of a sign language interpreter from the 
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sign language interpreter registration table, a function to 
call the videophone terminal -i ^of a sign language intcrprotorQ 
interpreter by using the extracted terminal number of the sign 
language — interpreter , and a function to call the called 
videophone terminal by using the acquired called terminal 
number_^ and 

video/ audio communications means including a function 

to synthesize at least a video from the videophone terminal 
for of the non-deaf -mute persons- and a video from the videophone 
terminal && Bof the sign language interpreters- and transmit the 
resulting video to the videophone terminal #e^ of the deaf-mute 
persons-, a function to transmit at least a video from the 
videophone terminal of the deaf-mute person and an audio from 
the videophone terminal of the sign language interpreter to 
the videophone terminal for the non-deaf-mute person and a 
function to transmit at least a video from the videophone 
terminal of the deaf-mute persone and an audio from the 
videophone terminal of the videophone — terminal — fe^^ — sign 
language interpretero non-deaf -mute person to the videophone 
terminal for non deaf mute pcrGons, and a function to transmit 
at least a video from of the videophone terminal for deaf mute 
porgons — aftd — a^i — audio — from — t4=ie — videophone — terminal — fe^^ 
non deaf mute — perGons — t-e — 14=5« — videophone — terminal — for sign 
language interpreters-. 

With in this configuration, the deaf mute person can have 
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Q vidcophono conversation with a non deaf mute pcrGon via sign 
language interpretation by uoing the video input/output device 

for convcrGation with — sign language. — 

ft-s -nianner , a function of thio invention is provided to 

extract and call the terminal number of a sign language 
interpreter registered in a sign language interpreter 

registration table — 3rs — included, A sign language 

interpreter can prcGcnt a provide sign language interpretation 
anywhere he/she may be, as long as he/she eea — be — called, 
thereby h as access to a videophone terminal- This provides a 
flexible and efficient sign language interpretation system can 
be provided . 

The invention dcGcribed in claim 10 is the sign language 

interpretation Gystcm according to claim 9, characterized in 
that Gclection Selection information for selecting a sign 
language interpreter is preferably registered in the sign 
language interpreter registration table_^ and that — fei=tethe 
terminal connection means includes a function to acquire the 
conditions for selecting a sign language interpreter from the 
calling videophone terminal and a function to extract the 
terminal number of a sign language interpreter who satisfies 
the acquired selection conditions for the sign language 
interpreter from the sign language interpreter registration 
table. 

With this configuration, a sign language interpreter who 
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satisfies the object of the conversation over a videophone 
between a deaf-mute person and a non-deaf -mute person from 
among the sign language interpreters registered in the sign 
language interpreter registration table can bo is selected . 

The sign language interpreter registration table 
includcG — has an availability flag to register whether a 

registered sign language interpreter is available-; ¥fee 

controller rGfcreneeo an and the control means has a function 
to reference the availability f lag f lags in the sign language 
interpreter registration table to extract the terminal number 
of an available sign language interpreter. It is thus possible 
to automatically select an available sign language interpreter-r 
thereby — eliminating — uGcless . This eliminates unnecessary 
calling and providing p r ovides a more flexible and efficient 
sign language interpretation system. 

The above object^ — other — ob j oct s , — char act or is tics — aftd: 

advantagcG of the invention will be apparent from the following 
detailed deocription of the embodiments of the invention made 
referring to drawings. 

Brief Description of the Dravjingo 

Fig. 1 is a system bloclc diagram of a video input /output 

device — for conversation v^ith sign language — according to an 
embodiment of the invention; 

Fig- 2 shows examples of a video displayed on the terminal 
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of the opponent party of a convcroation v<yith sign language by 
v^ay of the video input /output device for convcraation with sign 

language according to the invc ntion; The terminal 

connection means preferably includes a function to register 
a term in the term registration table via an operation from 
a videophone terminal, a function to select a term to be used 
from the terms registered in the term registration table via 
an operation from a videophone terminal, a function to generate 
a telop of the selected term, and a function to synthesize the 
generated telop onto a video to be transmitted to the opponent 
P^^ty. 

This makes it possible to display, in a telop, on the 

videophone terminal of the conversation partner a term that 
is hard to explain with sign language during sign language 
interpretation or a word that is hard to pronounce , 

The above and other features, elements, steps, 

characteristics and advantages of the present invention will 
be apparent from the following detailed description of 
preferred embodiments of the invention made referring to 
drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a system block diagram of a video input/output 

device for conversation with sign language according to a 
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preferred embodiment of the present invention; 

Fig. 2 shows examples of a video displayed on the terminal 

of a party of a conversation with sign language via the video 
input/output device for sign language conversation according 
to a preferred embodiment of t he present invention; 

Fig. 3 is a system block diagram of a sign language 
interpretation system according to a ^a preferred embodiment 
of the present i nvention; 

Fig. 4 shows an example of a video displayed on each screen 
of a deaf-mute person terminal, non-deaf -mute person terminal , 
and sign language interpreter terminal in sign language 
interpretation using the sign language interpretation system 
according to ^oh ^a preferred embodiment of the present 
invention; 

Fig. 5 is a process-iftf^ flowchart of a controller in a 
sign language interpretation system according to e fta preferred 
embodiment of th ^the present i nvention; 

Fig. 6 shows an example of a sign language interpreter 
registration table ; 

Fig. 7 shows an example of a screen for prompting input 
of a called terminal number; 

Fig. 8 shows an example of a screen for prompting input 
of sign language interpreter selection conditions; 

Fig. 9 shows an example of a screen for displaying a list 
of sign language interpreter candidates; 
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Fig- 1 0 is a system block diagram of a sign language 
interpretation system according to another preferred 
embodiment of the present i nve ntion; 

Fig. 11 shows an example of a connection table; 

Fig. 1 2 is a processing flowchart of the connection 
processing of a sign language interpretation system according 
to another preferred embodiment of the present invention ; 

Fig. 13 is a conceptual diagram showing a conversation 
with sign language between deaf-mute persons by using a prior 
art videophone terminal; and 

Fig. 1 4 is a conceptual diagram of a sign language 
interpretation service using a prior art multipoint connection 
unit . 

Boot Mode for Carrying Out the Invention 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
Fig. 1 is a system block diagram of a video input/output 
device for sign language conversation with — sign — language 
according to e ^a preferred embodiment of the present invention . 
In Fig. 1, a — numeral 12 represents a display device for 
displaying a sign language video, numeral 1 3 represents a 
fixture for fixing the display device 1 2 in front of the eyes 
of a deaf-mute person, numeral 14 represents a sign language 
imaging camera for picking up the sign language of the deaf-mute 
person, numeral 1 5 represents a waist fixture for fixing the 
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sign language imaging camera 14 at the waist of the deaf-mute 
person, numeral 16 represents a target imaging camera for 
picking up a target other than sign language, numeral 1 7 
represents a video synthesizer for synthesizing a video from 
the sign language imaging camera 14 and a video from the target 
imaging camera 16, numeral 1 8 represents a videophone 
connection device for connecting the display device 1 2 and the 
video synthesizer 17 to a videophone terminal 10. 

The display device 12 useSj;_ for example_^ a small- sized 
liquid crystal display having a sufficient resolution to 
display a sign language video. The display device 12 enlarges 
a video s-e — such that a deaf-mute person can recognize sign 
language displayed with the fixture 13 attached. G ftA convex 
lens is attached on the surface of the display device 1 2— 3rS- 
attached a convex lens so , such that sign language displayed 
on the display device 12 is oubotant ially — brought into 
substantial focus while the deaf-mute person is viewing the 
outer world_^ such as_^ the conversation partner and the scenery. 
This allowo enables the deaf-mute person to normally recognize 
the sign language displayed on the display device 1 2 while 
viewing the outer world. 

The fixture 13 ha ^includes a spectacle frame structure 
which can be fixed to the ears and nose of a deaf-mute person. 
Near the frame in front of the eyes of the deaf-mute person 
is attached the display device 1 2 is attached for viewing of 
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sign language without impairing the sight of the outer world. 
While the display device 1 2 is provided in a_lower left position 
in front of the eyes of the deaf-mute person in this example, 
-3rfe — the display device 1 2 may be provided anywhere as long as 
it does not impair the sight of the outer world. 

While the display devices 1 2 are provided on the same 
right and left positions of the fixture 13 oo go to such that 
the deaf-mute person can more clearly rocognizG — view the 
displayed sign language in this example, the display unit 12 
may be provided on either side of the fixture 1 3 as long as 
the deaf-mute person can rcGognizc view the displayed sign 
language . 

The fixture 1 3 is used provided to set the display device 
12 in front of the eyes of the deaf-mute person, sesuch that 
the display device 12 may be fixed to a hollow frame.' Or, a 
transparent plate may be provided in a frame and the display 
unit 12 may be stuck adhered to the transparent plate. 
caGQ Where the deaf-mute person has myopia, hyperopia, 
astigmatism, or presbyopia_^ and thus — noGds , requires a 
corrective lens, a corrective lens may be provided in a frame 
and the display device 1 2 may be otucJc adhered to the corrective 
lens . 

The sign language imaging camera 1 4 v^hich may bo , such 
as a small-sized CCD camera_j_ is fixed to the waist fixture 15. 
In this practice, — the The sign language imaging camera 14 is 
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set to an angle of view that is wide enough to pick up capture 
the sign language of the deaf-mute person while it is being 
fixed to the waist fixture 15. 

The waist fixture 15 is^ for example^ a belt to fix the 
imaging camera 14a at the waist of a deaf-mute person. Any 
waist fixture may be used whooc as long as the waist fixture 
includes a buckle fea- &having an arm for fixing the sign language 
imaging camera 1 4 thereto so as to allow enable the sign language 
imaging camera 14 to be set in an orientation whorc such that 
the sign language of the deaf-mute person can be picked 
^^p-:- captured. This makes it possible to stably piclc up capture 
the sign language of the deaf-mute person by using the sign 
language imaging camera 14^ even when the deaf-mute person 
changes his/her position or orientation. 

The target imaging camera 16 which may be similarly a 
such as a small-sized CCD camera_^ is fixed to the side of the 
fixture 13. When the deaf-mute person wears the fixture 13, 
the azimuth of imaging by the target imaging camera 16 is 
substantially the same as the direction of sight the line of 
sight of the deaf-mute person. This precisely captures the 
target for conversation precisely for transmission of the video 
obtained- 

The video synthesizer 17 synthesizes a target video from 
the target_^ imaging camera 16 and the sign language video from 
the sign language imaging camera 14 into a single synthesized 
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video. Several methods for synthesis shown in Fig. 2 are 
available-? — a. A method may be selected therefrom depending 
on the purpose. Fig. 2(a) is a Picture-in-Picture 
representation where the target video is shown as a main window 
and the sign language video is shown as a sub window. On the 
other hand, Fig. 2(b) is a Picture-in-Picture representation 
where the sign language video is shown as a main window and 
the target video is shown as a sub window. Fig. 2(c) is a 
Picture- in-Picture representation where the target video and 
sign language videos are displayed in equal size. Fig. 2(d) 
shows the sign language video alone. Fig. 2 (e) shows the target 
video alone. Fig. 2(f) is a Picture-in-Picture representation 
where a still picture with of the target video frozen is shown 
as a main window and the sign language video is shown as a sub 
window- On the other hand, Fig. 2(g) is a Pictur e-in-Picture 
representation where the sign language video is shown as a main 
window and a still picture v^^ith of the target video frozen is 
shown as a sub window. 

Sotting The setting of the position of the sub window 

in a Picture-to-Picture representation is preferably subject 
to change as required so as not to masjc obstruct the view of 
important information in a main window or hide another sub 
window inserted in sign language interpretation described 
later . 

The video synthesizer 1 7 may be accommodated in the waist 
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fixture 15 or fixture 1 3 so as to supply a video signal from 
the target imaging camera 16 or sign language imaging camera 
1 4 to the video synthesizer 1 7 accommodated in the waist fixture 
15 or fixture 13 over a wired or wireless connection . 

The videophone connection device 18 is a device which 
connects the display device 12 and the video synthesizer 17 
with the external device connecting terminal of the videophone 
terminal 10. The videophone connection device 18 supplies a. 
video signal being received by the videophone terminal 10 to 
the display device 12 qg well gg , and supplies a video signal 
from the video synthesizer 17 to the videophone terminal 10. 
Thus^ the display device 1 2 acrvco ao defines an external video 
display device of the videophone terminal 1 0 and the target 
imaging camera 1 6 and the sign language imaging camera 14 GorvQ 
re defines external video input devices of the videophone 
terminal 10. 

When such a video input /output device for sign language 
conversation with sign language is connected to a videophone 
terminal and a sign language conversation with oign language 
is initiated, the deaf-mute person can transmit a target video 
along with a sign language explanation by gign language addcd of 
the target video to the conversation partner. This provides 
the same advantagc advantages as that obtained by an unimpaired 
person ''s aural explanation added to of the target video. As 
a result, a opcQdy shorter conversation is made 
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poQoiblo . required. Further, it is possible to transmit 
information oa about the target to the opponent 
party conversation partner in a more efficient and precise 
fashion, m anner . 

While the fixture 13 for fixing the display device 12 
in front of the eyes of a deaf-mute person uses a spectacle 
frame structure in the above -described preferred embodiment, 
the fixture 1 3 may compriGc include a hair band fixed to the 
head equipped with an arm for supporting the display device 
12, or may have any suitable structure as long as it can fix 
the display device 1 2 can be fixed in front of the eyes of the 
deaf-mute person. 

While the target imaging camera 1 6 is fixed to the side 
of the fixture 1 3 in the above Qxamplo, tho -described preferred 
embodiment, the present invention is not limited thereto but 
the . The target imaging camera 16 may be fixed to the head 
of the deaf-mute person separately from the fixture 13. 

While the sign language imaging camera 14 compriGCS 
includes the waist fixture 15 fixed at the waist of the 
deaf-mute person in the above -described preferred embodiment, 
the sign language imaging camera 1 4 may use any type of fixing 
mcanG device as long as it can piclc up capture the sign language 
of the deaf-mute person and providco the same offoct of the 
invention . 

While the target imaging camera 16 for picking 
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^capturing a target for a conversation other than sign 
language is provided in the above -described preferred 
embodiment, an external video signal input terminal for 
inputting external video signal may be provided and a video 
signal input from the external video signal input terminal and 
a video signal from the sign language imaging camera 14 may- 
be synthesized by the video synthesizer 17 for transmission 
to the conversation partner. With this configuration, it is 
possible to display a video from an external camera or a video 
from a VTR as a target for the conversation and discussion with 
the partner efi rabout the contents of the video v^^ith via sign 
1 anguage ig allowed . 

While the videophone connection device 18 connects the 
display device 1 2 and the video synthesizer 1 7 with the external 
device connecting terminal of the videophone terminal 10, via 
wires in the above -described preferred embodiment, a radio 
communications device for wirelessly transmitting/receiving 
a video signal may be provided on each of the external device 
connecting terminal of the videophone terminal 10, the fixture 
13 and the video synthesizer 17. This eliminates the need for 
cabling cables to be connected to the videophone terminal 10, 
the fixture 13, and the video synthesizer 17, which provides 
QxtrcmQ casG of f acilitates han dling of the device . 

In case Where the videophone terminal 1 0 compriGCG 

a includes a wireless interface conforming to a 
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standard s tandard such as Bluetooth® for communicating with an 
external device, a communications device conforming to the same 
Standard s tandard should be provided on each of the fixture 1 3 
and the video synthesizer 17. By doing so, it is possible to 
communicate a video signal without physically connecting 
anything to the videophone terminal 10 as long as the 
communications devices provided on the fixture 1 3 and the video 
synthesizer 17 are within the service area of the wireless 
interface of the videophone terminal 10, which adds to the caao 
e #f urther facilitates handling. 

While a videophone terminal of a telephone type, 
especially a videophone terminal of a cellular phone type is 
used in the above -described preferred embodiment, the present 
invention is not limited thereto but a . A videophone terminal 
of the IP type to connect to the Intornct internet may also be 



While the above -de scribed preferred embodiment 
describes a video input /output device for conversation with 
sign language compriging — a — conversation including a sign 
language imaging camera 14, a target imaging camera 16, a video 
synthesizer 17, a display device 12, a fixture 13, and a 
videophone connection device 18, charactGriEcd in that w herein 
the video input /output device for convorgat ion — with — sign 
language conversation includes both a function to synthesize 
a sign language video and a target video and supplying the 




-used. 
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resulting video to the videophone terminal 10 and a function 
to acquire a sign language video being received by a videophone 
terminal 1 0 and display the sign language video on the display 
device 12, a video input device for convorQation v ^ zith sign 
language compriaing a conversation including a sign language 
imaging camera 1 4 for picking up sign language, a target imaging 
camera 16 for picking up a target other than sign language, 
a video synthesizer 17 for synthesizing a video from the sign 
language imaging camera 14 and a video from the target imaging 
camera 16, and a videophone connection device 18 for supplying 
the synthesized video signal to the videophone terminal 10 
allows the deaf-mute person to perform provide a sign language 
explanation by oign language while transmitting the video of 
a target other than sign language to the opponent party, which 
providoo the offcct of — the invention, conversation partner. 

Next, a sign language interpretation system will be 
described which allows enables selection of a sign language 
interpreter satisfying the object of a conversation in case w hen 
a deaf-mute person converses with a non-deaf -mute person via 
a sign language interpreter by using a video input/output 
device for conversation with sign language according to the 
invention . conversation . 

Fig. 3 is a system block diagram of a sign language 
interpretation system according to a ^a preferred embodiment 
of the invention. In Fig. 3, a-numeral 100 represents a sign 
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language interpretation system installed in a sign language 
interpretation center which provides a sign language 
interpretation service. The sign language interpretation 
system 100 interconnects, via a public telephone line 40, a 
videophone terminal for deaf-mute persons used by a deaf-mute 
person A (hereinafter referred to as a deaf-mute person 
terminal) 10, a videophone terminal for non-deaf -mute persons 
used by a non-deaf -mute person B (hereinafter referred to as 
a non-deaf -mute person terminal) 20, and a videophone terminal 
for sign language interpreters used by a sign language 
interpreter C (hereinafter referred to as a sign language 
interpreter terminal) 30 in order to provide a sign language 
interpretation service in a videophone conversation between 
a deaf-mute person and a non-deaf -mute person. In this 
preferred embodiment , each of the deaf-mute person terminal 
10, non-deaf -mute person terminal 20 and sign language 
interpreter terminal 30 is preferably a telephone-type 
videophone terminal to be connected to a public telephone line, 
and in particular_^ a wireless videophone terminal of the 
cellular phone type which can be carried for use on the road . 

Such a videophone terminal connected to a public line 
may be an ISDN videophone terminal based on ITU-T 
recommendation H.320, the present invention is not limited 
thereto and may use a videophone terminal which cmploys uses 
a unique protocol. 
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when the video input/output device for conversation with 
sign language is connected to the deaf-mute person terminal 
1 0 and the deaf-mute person A wears the fixture 1 3 and the waist 
fixture 15, a sign language video received by the deaf-mute 
person terminal 10 is displayed on the display device 12 fixed 
in front of the eyes of the deaf-mute person A. The target 
imaging camera 16 for picking up the area in the direction of 
sight line of the deaf-mute person A and the sign language 
imaging camera 14 for picking up the sign language of the 
deaf-mute person are set and a synthesized video including a 
video of the target and explanation by sign language is 
transmitted to the opp oncnt o ther party . 

The non-deaf-mute person terminal 20 is a general 
videophone terminal comprising — including a video display 
section 20a for displaying a video received from the 
opponent other party, an imaging section 20b for picking up the 
user or target, and a headset 20c for audio input /output . 

The sign language interpreter terminal 30 is also a 
general videophone terminal e# having a similar configuration 
to the non-deaf -mute person terminal 20, except that the video 
display section 30a is mainly primarily used to view the sign 
language of the deaf-mute person A and the video imaging section 
30b is mainly primarily used to pick up the sign language 
translated into by the sign language interpreter. The headset 
30c is mainly primarily used to listen to the voice of the 
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non-deaf -mute person B and to input the voice translated into 
from the translation of the sign language of the deaf-mute 
person A. 

While input/output of voice is made using a handset on 
a typical telephone- type terminal, a headset is used instead 
in order to keep free both hands of the user whose major concGrn 
jr &who performs sign language free . In the following 
description, a terminal uses a headset fixed on the head of 
the user including a non-deaf -mute person B. While a headset 
is not shown on the deaf-mute person terminal 10, a headset 
may be used and voice communications may also be used as vjcll , 
in caGQ in situations where a helper is there . present . | 

The sign language interpretation system 100 
GomprisGG includes a line interface for the deaf mute person 
terminal to connect 120 that is connected to a deaf-mute person 
terminal (hereinafter referred to as an I/F) 120, a line I/F 
for the non deaf mute- person terminal 140 to connect that is 
connected to a non-deaf -mute person terminal, and a line I/F 
for the 160 that is connected to a sign language interpreter 
terminal 160 to connect to a sign language interpreter terminal 
To each I/F are connected a . A multiplexer/demultiplexer 122, 
142, 162 for multiplexing/demultiplexing a video signal is 
connected to each of the line I/F 1 20 , 140, 1 60 , an audio signal 
or a data signal, a video CODEC (coder/decoder) 1 24, 144, 1 64 
for compressing/expanding a video signal, and an audio CODEC 
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1 26, 146, 1 66 for compressing/expanding an audio signal . Each 
line I/F, each multiplexer/demultiplexer, and each video CODEC 
or each audio CODEC perform call control, streaming control 
compression/expansion of a video/audio signal in accordance 
with a protocol used by each terminal. 

¥e -A video synthesizer 1 28 is connected to the video input 
of the video CODEC 124 for the deaf-mute person terminal 124 
io GonnQctod a video aynthogigcr 1 2 8 for synthesizing the video 
output of the video CODEC for the non-deaf -mute person terminal 
144, the video output of the video CODEC for the sign language 
interpreter terminal 164 and the output of the telop memory 
for the deaf-mute person terminal 132. 

¥e — An audio synthesizer 130 is connected to the audio 
input of the audio CODEC for the deaf-mute person terminal 126 
is connoctod an audio synthesizer 130 for synthesizing the 
audio output of the audio CODEC for the non-deaf -mute person 
terminal 146 and the audio output of the audio CODEC for the 
sign language interpreter terminal 166. 

While audio input/output is not made generally provided 
on a deaf-mute person terminal in gcncrQl, — so that the audio 
CODEC 126 or the audio Gynthcsizcr 130 for the doaf mute person 
terminal may be omitted , a voice communications function is 
preferably provided for a case where in situations in which 
the environment sound of a deaf-mute person terminal is to be 
transmitted to a non-deaf-mute person terminal or a case where 
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a helper assists the deaf-mute person. 

¥e -A video synthesizer 1 48 is connected to the video input 
of the video CODEC for the non-deaf -mute person terminal 144 
i3 connected q video oynthcGizcr 1 48 for synthesizing the video 
output of the video CODEC for the deaf-mute person terminal 
124, the video output of the video CODEC for the sign language 
interpreter terminal 164 and the output of the telop memory 
for the non-deaf-mute person terminal 152. 

¥ eAn audio synthesizer 150 is connected to the audio 
input of the audio CODEC for the non-deaf -mute person terminal 
146 is Gonncctcd an audio oynthcGigor 150 for synthesizing the 
audio output of the audio CODEC for the deaf-mute person 
terminal 1 26 and the audio output of the audio CODEC for the 
sign language interpreter terminal 166. 

While video display of a sign language interpreter may 
be omitted on a non-deaf-mute person terminal, understanding 
of the voice interpreted by the sign language interpreter is 
made — caoy f acilitated by displaying the video of the sign 
language interpreter, -se such that a function is preferably 
provided to synthesize the video of a sign language 
interpreter . 

video synthesizer 1 68 is connected to the video input 
of the video CODEC for the sign language interpreter terminal 
164 is connected a video synthoGizcr 168 for synthesizing the 
video output of the video CODEC for the deaf-mute person 
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terminal 124, the video output of the video CODEC for the 
non-deaf-mute person terminal 144 and the output of the telop 
memory for the sign language interpreter terminal 172. 

¥e — An audio synthesizer 170 is connected to the audio 
input of the audio CODEC for the sign language interpreter 
terminal 166 is — connQctcd — aa — audio — oynthcGizcr — 1-^?-0 — for 
synthesizing the audio output of the audio CODEC for the 
deaf-mute person terminal 126 and the audio output of the audio 
CODEC for the non- deaf -mute person terminal 146. 

While video display of a non-deaf -mute person may be 
omitted on a sign language interpreter terminal, understanding 
of the voice in interpreting the voice of a non-deaf -mute person 
is made — easy facilitated by displaying the video of the 
non-deaf -mute person, sesuch that a function is preferably 
provided to synthesize the video of a non-deaf -mute person. 

The sign language interpretation system 100 is equipped 
with a sign language interpreter registration table 1 82 whore ^ 
in which the terminal number of a terminal for sign language 
interpreters used by a sign language interpreter is registered 
and includes a controller 180 connected to each of the line 
I/FS 1 20, 140, 1 60 , multiplexers/demultiplexers 1 22, 144, 1 62, 
video synthesizers 1 28 , 1 48, 1 68, audio synthesizers 1 3 0 , 1 50, 
170, and telop memories 132, 152, 172. The controller 180 
providoG a function to connoct connect s a calling terminal, a 
sign language interpreter terminal and a called terminal by 
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v/ay of v ia a function to accept a call from a terminal used by 
a deaf-mute person or a terminal used by a non-deaf -mute person , 
a function to prompt a calling terminal to enter the called 
terminal number, a function to extract the terminal number of 
a sign language interpreter from the sign language interpreter 
registration table 182, a function to call the extracted 
terminal number, and a function to call the terminal number 
of the called terminal, and also provides a function to switch 
between video/audio synthesis methods used by video/audio 
synthesizers and a function to generate a telop and transmit 
the telop to a telop memory. 

Figs. 4 — ohov^G (a) - 4 (c) show an example of a video 
displayed on the screen of each terminal during a videophone 
conversation by way of via the sign language interpretation 
system according to a preferred embodiment of the present 
invention. Fig. 4(a) shows the screen of a deaf-mute person 
terminal. A video synthesizer 128 displays on the screen a 
video obtained by synthesizing a video of a non-deaf -mute 
person terminal and a video of a sign language interpreter 
terminal. While the video of the non-deaf -mute person is 
displayed as a main window and the video of the sign language 
interpreter is displayed as a sub window in a 
Picture-in-Picture fashion, a Pic ture-in-Pic ture display -ars- 
aloo poGoiblc QGGuming in which the video of the sign language 
interpreter e^s — a— is the main window and the video of the 
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non-deaf -mute person as a sub window-; — is also possible. 
Alternatively , these videos may be displayed -arfv -so as to have 
an equal size. When the video of a sign language interpreter 
is displayed in a larger size, the sign language interpreted 
by the sign language interpreter is easier to view and 
understand. A command from a terminal is preferably used to 
change the position of a sub window in the Picture-in-Picture 
display -&e such that the sub window will not mask obstruct the 
view of important information in the main window- 
Fig. 4(b) shows the screen of a non-deaf -mute person 
terminal. The video synthesizer 148 displays on the screen 
a video obtained by synthesizing a video of a deaf-mute person 
terminal and a video of a sign language interpreter terminal. 
While the video of the deaf-mute person terminal is a 
Picture-in-Picture representation including the target video 



language video picked up captured by the sign language imaging 
camera 14 arranged on the lower left of the target video, and 
the video of the sign language interpreter arranged on the lower 
right of the target video. The video of the sign language 
interpreter may be omitted. By displaying the video of the 
sign language interpreter in a Picture-in-Picture fashion, the 
non-deaf-mute person can chccic observe the expression of the 
sign language interpreter on the screen, which malccs it caaicr 
^ta undcrstand f acil itates understanding of the voice 




^captured by the target imaging camera 16, the sign 
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translated into sign language by the sign language interpreter . 

Fig. 4(c) shows the screen of a sign language interpreter 
terminal. The video synthesizer 168 displays on the screen 
a video obtained by synthesizing a video of a deaf-mute person 
terminal and a video of a non-deaf -mute person terminal, ift 
thia GQGQ qIgo, the The video of the deaf-mute person terminal 
is a Picture-in-Picture representation including the target 
video picked up captured by the target imaging camera 16, the 
sign language video picJced up captured by the sign language 
imaging camera 1 4 arranged on the lower left of the target video, 
and the video of the non-deaf -mute person arranged on the lower 
right of the target video. The video of the non-deaf -mute 
person may be omitted. By displaying the video of the 
non-deaf -mute person in a Picture-in-Picture fashion, the sign 
language interpreter can check observe the expression of the 
non-deaf-mute person on the screen, which makcg it oasior to 
undorGtand the facilitates understanding of the voice of the 
non-deaf-mute person as a target for sign language 
interpretation . 

order te To support a gqgo where t-he 

environment situation in which the environmental sound of a 
deaf-mute person terminal is to be transmitted or a case 
where situation in which a helper assists the deaf-mute person, 
a voice obtained by synthesizing the voice from the 
non-deaf -mute person terminal and the voice from the sign 
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language interpreter terminal by using the audio synthesizer 
1 30 is output to the deaf-mute person terminal , a voice obtained 
by synthesizing the voice from the deaf-mute person terminal 
and the voice from the sign language interpreter terminal by 
using the audio synthesizer 150 is output to the non-deaf -mute 
person terminal, and a voice obtained by synthesizing the voice 
from the non-deaf -mute person terminal and the voice from the 
deaf-mute person terminal by using the audio synthesizer 170 
is output to the sign language interpreter terminal. 

In CQSQ When it is not necessary to transmit the 

environment e nvironmental sound of the deaf-mute person 
terminal or a helper is not present, the audio synthesizers 
130, 150 and 170 may be omitted and the output of the audio 
CODEC for the non-deaf -mute person terminal 146 may be 
connected to the input of the audio CODEC for the sign language 
interpreter terminal 166 and the output of the audio CODEC for 
the sign language interpreter terminal 166 may be connected 
to the input of the audio CODEC for the non-deaf -mute person 
terminal 146. 

Operation of the video synthesizers 128, 148, 168 and 
audio synthesizers 1 30, 1 50, 1 70 is controlled by the 
controller 180. The user may change the video output method 
or audio output method by pressing a predetermined number 

button ef — on a dial pad of each terminal. This is 

i - mp 1 cmon t od i nitiated when a push on the number button on the 
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dial pad of each terminal is detected as a data signal or a 
tone signal by the multiplexer/demu Itiplexer 1 22, 1 44, 162 and 
detection of the push on the button is signaled to the 
controller . 

With this configuration, flexibility in the usage of the 
system on each terminal is ensured- For example, only 
necessary videos or audios are selected and displayed/output 
in accordance with the object or it is possible to replace a 
main window with a sub window, or change the position of the 
sub window. 

To the input of the audio gynthcGizorG 1 28, — 1 4 8, — 1 68 are 
rQopcGtively connocted q A telop memory for the deaf-mute person 
132, a telop memory for the non-deaf -mute person 152, and a 
telop memory for the sign language interpreter 172 are 
respectively connected to the input of the audio synthesizers 
128, 148, 168 . Contents of each telop memory 132, 152, 172 
can be are set fromb y the controller 180. 

With this configuration, by setting a message to be 
displayed on each terminal to the telop memories 1 32, 1 52, 1 72 
and issuing an instruction to select a signal of the telop 
memories 1 32, 1 52, 1 72 to the audio synthesizers 1 28, 148, 1 68 
in the setup of a videophone conversation via sign language 
interpretation, it is possible to transmit necessary messages 
to respective terminals to establish a three-way call. 

In case there is situations in which a difficult term 
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which is hard to oxplQin must be explained using sign language 
or a word which is hard that is difficult to pronounce in a 
videophone conversation, it is possible to register in advance 
the term in the term registration table 184 of the controller 
180 in QSGociation with the corresponding to a number ej— on 
the dial pad on each terminal. By doing so, it is possible 
to detect a push on the dial pad on each terminal during a 
videophone conversation, extract the term corresponding to the 
number of the dial pad pressed from the term registration table , 
generate a text telop, and set the text telop to each telop 
memory, thereby displaying the term on each terminal. 

With this configuration, a term which is hard dif f icult 
to explain using sign language or a word which is hard dif f icult 
to pronounce is transmitted by way of via a text telop to the 
opponent party conversat ion partner , thus_^ providing a quicker 
and more to-the-point videophone conversation. 

Next , a processing flow of the controller 180 for setting 
a videophone conversation via sign language interpretation is 
explained. 

Prior to processing, information to select a sign 
language interpreter and the terminal number of a terminal used 
by each sign language interpreter are registered in the sign 
language interpreter registration table 182 of the controller 
180 from an appropriate terminal (not shown). Fig. 5 shows 
an example of a_regis tration item to be registered in the sign 
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language interpreter registration table 182. The information 
to select a sign language interpreter refers to information 
used by the user to select a desired sign language interpreter, 
which includes a— sex, eft— age, a-habitation, a— specialty, and 

the level of sign language interpretation-: The habitation 

skill. Habitation assumes a caoQ v^hQrc situation in which the 
user doGiroG w ants a person who has geographic knowledge eftof 
a specific area and, in this example, a ZIP code is used to 
specify an area. The opoGialty Special ty assumes a caGc whore , 
in caGO the convoraation pcrtaino to a gpccif ic field, situation 
in which the user dcGircG w ants a person who has expert knowledge 
on the in a particular field or is familiar with the topics in 
the that field. In this example, the fields in which a sign 
language interpreter is familiar with are classified into 
several categories to be registered, such as politics, law, 
business, education, science and technology, medical care, 
language, sports, and hobby. The specialties are diverse, 
sesuch that they may be registered hierarchically and searched 
through at a level desired by the user when selected. 

In addition, qualifications of each sign language 
interpreter may be registered in advance for the user to select 
a qualified person as a sign language interpreter. 

The terminal number to be registered is the telephone 
number of the terminal, because in this example a videophone 
terminal to connect to a public telephone line is 
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QGGumcd . p rovided , 

In the sign language interpreter registration table 182 
id provided , an availability flag is provided to indicate 
whether sign language interpretation can be accepted. A 
registered sign language interpreter can call the sign language 
interpretation center from his/her terminal and enter a command 
by using a dial pad to set /reset the availability flag. Thus, 
a sign language interpreter registered in the sign language 
interpreter registration table can set the availability flag 
only when he/she is available for sign language interpretation, 
thereby eliminating useless calling and allowing p ermit t ing 
the user to select an available sign language interpreter 
without delay - 

Fig. 5 shows a processi-i=ig flowchart of the controller 
180. The sign language interpretation system 100 allows a 
deaf-mute person terminal or non-deaf -mute person terminal to 
propose a sign language interpretation service. From the 
deaf-mute person terminal, the user places a call to a telephone 
number on the line I/F for the deaf-mute person terminal. From 
the non-deaf -mute person terminal, the user places a call to 
a telephone number on the line I/F for the non-deaf -mute person 
terminal. This calls the sign language interpreter terminal 
and the opponcnt par tner ^ s terminal and establishes a 
videophone connection via sign language interpretation. 

As shown in Fig. 5, it is firGt dctQctcd that the line 
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I/F for the deaf-mute person terminal 120 or line I/F for the 
non-deaf -mute person terminal 140 that is called is detected 
first. (SI 00) . Next, the calling terminal displays a screen to 
prompt input of the terminal number of the called party shown 
in Fig. 7 (SI 02 ) .Jxhe terminal number of the called party input 
by the caller is acquired (SI 04). The calling terminal 
displays a screen to prompt input of the selection conditions 
for a sign language interpreter shown in Fig. 8 (SI 06) . The 
sign language interpreter selection conditions input by the 
caller are acquired (SI 08). The sign language interpreter 
selection conditions input by the caller are sex, age bracket, 
area, specialty and sign language level. A corresponding sign 
language interpreter is selected based on the sex, age, 
habitation, specialty, and sign language level registered in 
the sign language interpreter registration table 182. The 
area is specified by using a ZIP code and a sign language 
interpreter is selected starting with the habitation closest 
to the specified area. For any selections, in cagc if it is 
not necessary to specify a condition, N/A may be selected. 

Next, a sign language interpreter with availability flag 
set is selected from among the sign language interpreters 
satisfying the selection conditions acquired referring to the 
sign language interpreter registration table 1 82 . The calling 
terminal displays a list of sign language interpreter 
candidates shown in Fig. 9 to prompt input of the selection 



43 



number of a desired sign language interpreter (S1 10). The 
selection number of the sign language interpreter input by the 
caller is acquired (SI 12) and the terminal number of the 
selected sign language interpreter is extracted from the sign 
language interpreter registration table 182 and the terminal 
is called (SI 14) . When the sign language interpreter terminal 
has accepted the call (S1 16), the called terminal number is 
extracted and called (8118). When the called terminal has 
accepted the call (SI 20) , a videophone conversation via sign 
language interpretation starts (SI 22). 

^ — caGc When the sign language interpreter terminal 
selected in SI 66 does not accept the call, whether a next 
candidate is available is determined (SI 24). In caso W hen a 
next candidate is available, execution returns to SI 14 and the 
procedure is repeated. Otherwise the calling terminal is 
notified as such and the call is released (SI 26). 

In casQ When the called terminal does not accept the 

call in SI 20, the calling terminal and the selected sign 
language interpreter terminal are notified as such and the call 
is released (SI 28). 

While in casc w hen the selected sign language interpreter 
terminal does not accept the call, the caller is notified es- 
such and the call is released in the above -described preferred 
embodiment, a sign language interpretation reservation table 
to register a calling terminal number and a called terminal 
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number may be provided and the caller and the called party may 
be notified on a later response from the selected sign language 
interpreter to set a videophone conversation. 

While the sign language interpretation system 100 
compri SCO includes a line I/F, a multiplexer/demultiplexer, a 
video CODEC, an audio CODEC, a video synthesizer, an audio 
synthesizer and a controller in the above -described preferred 
embodiment, these components need not be implemented provided 
by individual hardware (H/W)-) , but rather, the function of each 
component may be implemented based on provided via software 
running on a computer. 

While the sign language interpreter terminal 30, same 
es — the deaf mute person terminal — 1 0 and the non deaf mute 
person — terminal — — is located outside the sign language 
interpretation center and called from the sign language 
interpretation center over a public telephone line to provide 
a sign language interpretation service in the above -preferred 
embodiment, the present invention is not limited thereto but 
part , and a portion or all of the sign language interpreters 
may be provided in the sign language interpretation center to 
provide a sign language interpretation service from the sign 
language interpretation center. 

In the above -described preferred embodiment, a sign 

language interpreter eee may join a sign language 

interpretation service anywhere he/she may be, as long as 
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he/she has a terminal which can be connected to a public 
telephone line. Thus_j_ the sign language interpreter can 
provide a sign language interpretation service by using the 
availability flag to make efficient use of free time. By doing 
so, it is possible to stably operate a sign language 
interpretation service accompanied by a problem of difficult 
reservation of a sign language interpreter. In particular, 
the number of volunteer sign language interpreters is 
increasing nowadays. A volunteer who is available only 
irregularly can provide a sign language interpretation service 
by taking advantage of ertheir limited free time. 

While a video signal of the home terminal is not input 
to the video synthesizers 1 28, 148, 168 in the above - described 
preferred embodiment, a function may be provided to input the 
video signal of the home terminal for later synthesis and 
display to check the video on the terminal. 

While the video synthesizers 1 28, 1 48, 1 68 and the audio 
synthesizers 130, 150 170 are used to synthesize videos and 
audios for each terminal in the above -de scribed preferred 
embodiment, the present invention is not limited thereto , but 
videos . Video and audioo audio from all terminals may be 
synthesized at oncc the same time and the resulting video or 
audio may be transmitted to each terminal. 

While a function is provided whereby such that the telop 
memories 132, 152, 172 are provided and telops are added to 
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the video synthesizers 1 28, 148, 1 68 in order to display a text 
telop on each terminal in the above -de scribed preferred 
embodiment, a function may be^provided whereby a telop memory 
to store audio information and telops are added to the audio 
synthesizers 130, 150, 170 in order to output an audio message 
on each terminal. This makes it possible to set a videophone 
conversation via sign language interpretation even in case the 
non-deaf -mute person is a visually impaired person. 

Fig. 1 0 is a system block diagram of a sign language 
interpretation system according to another preferred 
embodiment of the present invention . This preferred 
embodiment shows a system configuration example aoouming w hich 
assumes that each terminal used by a deaf-mute person, a 
non-deaf -mute person and a sign language interpreter is an IP 
(Internet Protocol) type videophone terminal to connect to the 
Intornot internet equipped with a web browser. 

In Fig, 10, a numeral 200 represents a sign language 
interpretation system installed in a sign language 
interpretation center to provide a sign language 
interpretation service. The sign language interpretation 
system 200 connects a deaf-mute person terminal 50 used by a 
deaf-mute person, a non-deaf -mute person terminal 60 used by 
a non-deaf -mute person, and the selected sign language 
interpreter terminals used by a sign language interpreter 231 , 
232, ... via the Internet 70^ in order to provide a videophone 
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conversation service via sign language interpretation between 
the deaf-mute person and the n on-deaf -mute person. 

While each of the deaf-mute person terminal 50, the 
non-deaf -mute person terminal 60 and the sign language 
interpreter terminals 231, 232,.., each — compriGos includes a 
general-purpose processing device (a) such as a personal 
computer having a video input I/F function, an audio 
input/output I/F function and a network connection function, 
a keyboard (b) and a mouse (c) for input of information as well 
as a display (d) for displaying a web page screen presented 
by a web server 210 and a videophone screen supplied by a 
communications server 220, a television camera (e) for imaging 
the sign language of a sign language interpreter, and a headset 
(f) for performing audio input /output for the sign language 
interpreter, the processing device has -includes IP videophone 
software and a web browser installed in this example, a 
dedicated videophone terminal may be used instead. 

The videophone terminal connected to the 
Inter not internet may be an IP videophone terminal based on 
ITU-T recommendation H.323, the present invention is not 
limited thereto but , and may use a videophone terminal which 
employs operates according to a unique protocol. 

The Internet inter net may be e#-a wireless LAN typo . The 
videophone terminal may be a cellular phone or a portable 
terminal equipped with a videophone function and also including 



48 



* * 



a web access function. 

The sign language interpretation system 200 
GomprioGO : includes a communications server 220 including a 
connection table 222 for setting the terminal addresses of a 
deaf-mute person terminal, a non-deaf -mute person terminal and 
a sign language interpreter terminal as well as a function to 
interconnect the terminals registered in the connection table 
222 and synthesize a video and an audio received from each 
terminal and transmit the synthesized video and audio to each 
terminal7-_j_ a web server 210 including a sign language 
interpreter registration table 212 for registering the 
selection information, terminal address and availability flag 
of a sign language interpreter as mentioned earlier, as well 
as a function to select a desired sign language interpreter 
based on an access from a calling terminal by using a web browser 
and set the terminal address of each of the calling terminal, 
called terminal and sign language interpreter terminal in the 
connection table 222 of the communications sGrvGr2 20 ; server 
220 , a router 250 for connecting the web server 210 and the 
communications server 220 to the Internet ; internet , and a 
plurality of sign language interpreter terminals 231, 232,..., 
23N connected to the communications server 220 via a network. 

Fig. 11 shows an example of a connection table 222. As 
shown in Fig. 1 1 , the terminal address of a deaf-mute person 
terminal, the terminal address of a non-deaf -mute person 
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terminal and the terminal address of a sign language 
interpreter terminal are registered as a set in the connection 
table 222. This provides a single sign language 

interpretation service. The connection table 222 is designed 
to register a plurality of such terminal address set depending 
on the throughput of the communications server 220, thereby 
simultaneously providing a plurality of sign language 
interpretation service s . 

While the terminal address registered in the connection 
table 222 is an address on the Internet internet and is generally 
an IP address, the present invention is not limited thereto 
bu ^/ and, for example_^ a name given by a directory server may 
be used. 

The communications server 220 performs packet 
communications using a predetermined protocol with the 
deaf-mute person terminal, non-deaf -mute person terminal and 
sign language interpreter terminal set to the connection table 
222 and provides, by way of software processing, the functions 
similar to those provided by a multiplexer /demultiplexer 122, 
142, 1 62, a video CODEC 1 24, 144, 1 64, an audio CODEC 1 26, 146, 
166, a video synthesizer 128, 148, 168, an audio synthesizer 
1 30, 1 50, 1 70 in the above -described sign language 
interpretation system 100. 

With this configuration, same as similar to the sign 
language interpretation system 100, prescribed videos and 
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audios are communicated between a deaf-mute person terminal, 
a non-deaf -mute person terminal and a sign language interpreter 
terminal, and a videophone conversation via sign language 
interpretation is established between the deaf-mute person and 
the non-deaf -mute person. 

While the sign language interpretation system 100 uses 
the controller 180 and the telop memories 132, 152, 172 to 
extract a term registered in the term registration table 184 
during a videophone conversation based on an instruction from 
a terminal and displays the term as a telop on the terminal, 
the same function may also be provided by viay of via software 
processing by the communications server 220 in this preferred 
embodiment also , A term specified by each terminal may be 
displayed as a popup message on the other terminal by way of 
the web server 210. Or, a telop memory may be provided in the 
communications server 220 e- esuch that a term specified by each 
terminal will be written into the telop memory via the web 
server 210 and displayed as a text telop on each terminal . 

While the sign language interpretation system 1 00 uses 
the controller 1 80 to interconnect a deaf-mute person terminal , 
a non-deaf -mute person terminal and a sign language interpreter 
terminal, the connection procedure is made by the web server 
210 in this preferred embodiment because each terminal has a 
web access function. 

Fig. 12 is a processing flowchart of a connection 
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procedure by the web server 210, The sign language 
interpretation system 200 also allows enables a deaf-mute 
person terminal or non-deaf -mute person terminal to 
propooc request a sign language interpretation service. A 
deaf-mute person or a non-deaf -mute person wishing to 
propoGe request a sign language interpretation service 
accesses the web server 21 0 in the sign language interpretation 
center fey-using a web browser to log in from each own terminal, 
which starts the acceptance of the sign language interpretation 
service . 

As shown in Fig. 12, the web server 210 first acquires 
the terminal address of a caller (S200) and sets the terminal 
address to the connection table 222 (S202) . Next, the web 
server delivers a screen to prompt input of the called terminal 
address similar to that shown in Fig. 7 to the calling terminal 
(S204) . The called terminal address input by the caller is 
acquired (S206) . The web server delivers a screen to prompt 
input of the selection conditions for a sign language 
interpreter similar to that shown in Fig. 8 to the calling 
terminal (S208) . The sign language interpreter selection 
conditions input by the caller are acquired (S210). 

Next, a sign language interpreter with an availability 
flag set is selected from among the sign language interpreters 
satisfying the selection conditions acquired from the sign 
language interpreter registration table 212. The web server 
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210 delivers a list of sign language interpreter candidates 
similar to that shown in Fig. 9 to the calling terminal to prompt 
input of the selection number of a desired sign language 
interpreter (S212) . The selection number of the sign language 
interpreter input by the caller is acquired and the terminal 
address of the selected sign language interpreter is acquired 
from the sign language interpreter registration table 212 
(S214). Based on the acquired terminal address of the sign 
language interpreter, the web server 210 delivers a calling 
screen to the sign language interpreter terminal (S216). ift 
CQGG lf the call is accepted by the sign language interpreter 
(S218) , the terminal address of the sign language interpreter 
is set to the Connoction connection table 222 (S220) . Next, 
the web server 210 delvers a calling screen to the called 
terminal based on the acquired called terminal address (3222) . 
In cagc lf the call is accepted by the called terminal (S224) , 
the called terminal address is set to the connection table 222 
(S226) . Then, a videophone conversation via sign language 
interpretation otarts b egins (S228) . 

In cQSC if the sign language interpreter terminal does 
not accept the call in S218, whether a next candidate is 
available is determined (S230) . In CQGC if a next candidate 
is available, the web server delivers a message to prompt the 
caller to select another candidate (S232) to the calling 
terminal, then and the execution returns to S214. In cQGc If 
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another candidate is not found, the calling terminal is 
notified as ouch (S234) and the call is released. 

In caGc lf the called terminal does not accept the call 
in S224, the calling terminal and the selected sign language 
interpreter terminal are notified as ouch (S236) and the call 
is released. 

While in case When the selected sign language 

interpreter terminal does not accept the call, the caller is 
notified sts — such — and the call is released in the above 
cmbodimQnt -described preferred embodiments , a sign language 
interpretation reservation table to register a calling 
terminal address and a called terminal address may be provided 
and the caller and the called party may be notified efi — of a 
later response from the selected sign language interpreter to 
set a videophone conversation. 

While the sign language interpreter terminal is located 
in the sign language interpretation system 200 of the sign 
language interpretation center in the above 
embodiment - described preferred embodiments , the present 
invention is not limited thereto but , and some or all of the 
sign language interpreter terminals may be provided outside 
the sign language interpretation center and connected via the 
Internet . 

In the above Qmbodimcnt -descr ibed preferred embodiments , 
the configuration of the sign language interpretation system 
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has been described for a caoc whore — situation in which a 
videophone terminal used by a deaf-mute person, a non-deaf -mute 
person or a sign language interpreter is a telephone- type 
videophone terminal connected to a public telephone line^,^ and 
a case v^hcrc situation in which the videophone terminal is an 
IP- type videophone terminal connected to the Internet, the 
telephone- type videophone terminal and the IP- type videophone 
terminal can communicate with each other by arranging a gateway 
to perform protocol conversion therebetween. A sign language 
interpretation system conforming to one protocol may be 
provided via the gateway to support a videophone terminal 
conforming to the other protocol. 

In this wa ymanner , the sign language interpretation 
system allows enables the user to enjoy or provide a sign 
language interpretation service anywhere he/she may be, as long 
as he/she has a terminal which can be connected to a public 
telephone line or the Intornct . internet . A sign language 
interpreter does not always have to visit a sign language 
interpretation center but can present a sign language 
interpretation from his/her home or a facility or site where 
a videophone terminal is located, or provide a sign language 
interpretation service by using a cellular phone or a portable 
terminal equipped with a videophone function. 

A person with the ability of sign language interpretation 
may wish to register in the sign language interpreter 
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registration table in the sign language interpretation center 
in order to provide a sign language interpretation service 
any t imc who n wh enever it is convenient to him/her. From the 
viewpoint of the operation of the sign language interpretation 
center, it is . not necessary to summon sign language 
interpreters to the center. This allowg provides efficient 
operation of the sign language interpretation center both in 
terms of time and costs . In particular , the number of volunteer 
sign language interpreters is increasing nowadays. The sign 
language interpretation service can be provided from a sign 
language interpreter's home, which facilitates reservation of 
a sign language interpreter. 

Industrial Applicabili ty 

As mentioned above, according to ^fe epref erred 
embodiments of the present invention, a deaf-mute person can 
■add — include an explanation with sign language while 
transmitting a target video other than sign language. It is 
thus possible to precisely explain the target to thereby 
speeding up a conversation. 

While the present invention has been described with 
respect to preferred embodiments^ it will be apparent to those 
skilled in the art that the disclosed invention may be modified 
in numerous ways and may assume many embodiments other than those 
specifically set out and described above. Accordingly, it is 
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intended by the appended claims to cover all modifications of 
the invention which fall within the true spirit and scope of 
the invention. 
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